Search

As the power consumption of HPC systems will be a primary constraint for exascale computing, a main objective in HPC communities is recently becoming to maximize power efficiency (i.e., performance per watt) rather than performance. Although programmers have spent a considerable effort to improve performance by tuning HPC programs at a code level, tunings for improving power efficiency is now required. In this work, we select two representative HPC programs (Graph500 and SDPARA) and evaluate how traditional code-level performance tunings applied to these programs affect power efficiency. We also investigate the impacts of the tunings on power efficiency at various operating frequencies of CPUs and/or GPUs. The results show that the tunings significantly improve power efficiency, and different types of tunings exhibit different trends in power efficiency by varying CPU frequency. Finally, the scalability and power efficiency of state-of-the-art Graph500 implementations are explored on both a single-node platform and a 960-node supercomputer. With their high scalability, they achieve 27.43 MTEPS/Watt with 129.76 GTEPS on the single-node system and 4.39 MTEPS/Watt with 1,085.24 GTEPS on the supercomputer.

The breadth-first search (BFS) is one of the most centric processing in graph theory. In this paper, we presented a fast, scalable, and energy-efficient BFS for a nonuniform memory access (NUMA)-based system, in which the NUMA architecture was carefully considered. Our implementation achieved performance rates of 175 billion edges per second for Kronecker graph with \(2^{33}\) vertices and \(2^{37}\) edges on two racks of a SGI UV 2000 system with 1,280 threads and the fastest entries for a shared-memory system in the June 2014 and November 2014 Graph500 lists. It also produced the most energy-efficient entries in the first and second (small data category) and third, fourth, fifth, and sixth (big data category) Green Graph500 lists on a 4-socket Intel Xeon E5-4640 system.

The semi definite programming (SDP) problem is one of the central problems in mathematical optimization. The primal-dual interior-point method (PDIPM) is one of the most powerful algorithms for solving SDP problems, and many research groups have employed it for developing software packages. However, two well-known major bottlenecks, i.e., the generation of the Schur complement matrix (SCM) and its Cholesky factorization, exist in the algorithmic framework of the PDIPM. We have developed a new version of the semi definite programming algorithm parallel version (SDPARA), which is a parallel implementation on multiple CPUs and GPUs for solving extremely large-scale SDP problems with over a million constraints. SDPARA can automatically extract the unique characteristics from an SDP problem and identify the bottleneck. When the generation of the SCM becomes a bottleneck, SDPARA can attain high scalability using a large quantity of CPU cores and some processor affinity and memory interleaving techniques. SDPARA can also perform parallel Cholesky factorization using thousands of GPUs and techniques for overlapping computation and communication if an SDP problem has over two million constraints and Cholesky factorization constitutes a bottleneck. We demonstrate that SDPARA is a high-performance general solver for SDPs in various application fields through numerical experiments conducted on the TSUBAME 2.5 supercomputer, and we solved the largest SDP problem (which has over 2.33 million constraints), thereby creating a new world record. Our implementation also achieved 1.713 PFlops in double precision for large-scale Cholesky factorization using 2,720 CPUs and 4,080 GPUs.

Breadth-first search (BFS) is one of the most fundamental processing algorithms in graph theory. We previously presented a scalable BFS algorithm based on Beamer's direction-optimizing algorithm for non-uniform memory access (NUMA)-based systems, in which the NUMA architecture was carefully considered. This paper presents our new implementation that reduces remote memory access in a top-down direction of direction-optimizing algorithm. We also discuss numerical results obtained on the SGI UV 2000 and UV 300 systems, which are shared-memory supercomputers based on a cache coherent (cc)-NUMA architecture that can handle thousands of threads on a single operating system. Our implementation has achieved performance rates of 219 billion edges per second on a Kronecker graph with 234 vertices and 238 edges on a rack of an SGI UV 300 system with 1,152 threads. This result exceeds the fastest entry for a shared-memory system on the current Graph500 list presented in November 2015, which includes our previous implementation.

The breadth-first search (BFS) is one of the most centric kernels in graph processing. Beamer's direction-optimizing BFS algorithm, which selects one of two traversal directions at each level, can reduce unnecessary edge traversals. In a previous paper, we presented an efficient BFS for a non-uniform memory access (NUMA)-based system, in which the NUMA architecture was carefully considered. In this paper, we investigate the locality of memory accesses in terms of the communication with remote memories in a BFS for a NUMA system, and describe a fast and highly scalable implementation. Our new implementation achieves performance rates of 174.704 billion edges per second for a Kronecker graph with 233 vertices and 237 edges on two racks of a SGI UV 2000 system with 1,280 threads. The implementations described in this paper achieved the fastest entries for a shared-memory system in the June 2014 and November 2014 Graph500 lists, and produced the most energy-efficient entries in the second, third, and fourth Green Graph500 lists (big data category).

We introduce a memory efficient implementation for the NVM-based Hybrid BFS algorithm that merges redundant data structures to a single graph data structure, while offloading infrequent accessed graph data on NVMs based on the detailed analysis of access patterns, and demonstrate extremely fast BFS execution for large-scale unstructured graphs whose size exceed the capacity of DRAM on the machine. Experimental results of Kronecker graphs compliant to the Graph500 benchmark on a 2-way INTEL Xeon E5-2690 machine with 256 GB of DRAM show that our proposed implementation can achieve 4.14 GTEPS for a SCALE31 graph problem with 231 vertices and 235 edges, whose size is 4 times larger than the size of graphs that the machine can accommodate only using DRAM with only 14.99 % performance degradation. We also show that the power efficiency of our proposed implementation achieves 11.8 MTEPS/W. Based on the implementation, we have achieved the 3rd and 4th position of the Green Graph500 list (2014 June) in the Big Data category.

NVM devices will greatly expand the possibility of processing extremely large-scale graphs that exceed the DRAM capacity of the nodes, however, efficient implementation based on detailed performance analysis of access patterns of unstructured graph kernel on systems that utilize a mixture of DRAM and NVM devices has not been well investigated. We introduce a graph data offloading technique using NVMs that augment the hybrid BFS (Breadth-first search) algorithm widely used in the Graph500 benchmark, and conduct performance analysis to demonstrate the utility of NVMs for unstructured data. Experimental results of a Scale27 problem of a Kronecker graph compliant to the Graph500 benchmark show that our approach maximally sustains 4.22 Giga TEPS (Traversed Edges Per Second), reducing DRAM size by half with only 19.18% performance degradation on a 4-way AMD Opteron 6172 machine heavily equipped with NVM devices. Although direct comparison is difficult, this is significantly greater than the result of 0.05 GTEPS for a SCALE 36 problem by using 1TB of DRAM and 12 TB of NVM as reported by Pearce et al. Although our approach uses higher DRAM to NVM ratio, we show that a good compromise is achievable between performance vs. capacity ratio for processing large-scale graphs. This result as well as detailed performance analysis of the proposed technique suggests that we can process extremely large-scale graphs per node with minimum performance degradation by carefully considering the data structures of a given graph and the access patterns to both DRAM and NVM devices. As a result, our implementation has achieved 4.35 MTEPS/W(Mega TEPS per Watt) and ranked 4th on November 2013 edition of the Green Graph500 list in the Big Data category by using only a single fat server heavily equipped with NVMs.

Breadth-first search BFS is an important graph analysis kernel. The Graph500 benchmark measures a computer's BFS performance using the traversed edges per second TEPS ratio. Our previous nonuniform memory access NUMA-optimized BFS reduced memory accesses to remote RAM on a NUMA architecture system; its performance was 11 GTEPS giga TEPS on a 4-way Intel Xeon E5-4640 system. Herein, we investigated the computational complexity of the bottom-up, a major bottleneck in NUMA-optimized BFS. We clarify the relationship between vertex out-degree and bottom-up performance. In November 2013, our new implementation achieved a Graph500 benchmark performance of 37.66 GTEPS fastest for a single node on an SGI Altix UV1000 one-rack and 31.65 GTEPS fastest for a single server on a 4-way Intel Xeon E5-4650 system. Furthermore, we achieved the highest Green Graph500 performance of 153.17 MTEPS/W mega TEPS per watt on an Xperia-A SO-04E with a Qualcomm Snapdragon S4 Pro APQ8064.

The breadth-first search (BFS) is one of the most important kernels in graph theory. The Graph500 benchmark measures the performance of any supercomputer performing a BFS in terms of traversed edges per second (TEPS). Previous studies have proposed hybrid approaches that combine a well-known top-down algorithm and an efficient bottom-up algorithm for large frontiers. This reduces some unnecessary searching of outgoing edges in the BFS traversal of a small-world graph, such as a Kronecker graph. In this paper, we describe a highly efficient BFS using column-wise partitioning of the adjacency list while carefully considering the non-uniform memory access (NUMA) architecture. We explicitly manage the way in which each working thread accesses a partial adjacency list in local memory during BFS traversal. Our implementation has achieved a processing rate of 11.15 billion edges per second on a 4-way Intel Xeon E5-4640 system for a scale-26 problem of a Kronecker graph with 226 vertices and 230 edges. Not all of the speedup techniques in this paper are limited to the NUMA architecture system. With our winning Green Graph500 submission of June 2013, we achieved 64.12 GTEPS per kilowatt hour on an ASUS Pad TF700T with an NVIDIA Tegra 3 mobile processor.

This paper discusses ohmic contact formation of Sn-based alloys on n-InSb. Most critical problem, we encountered when using a Sn/Au/Ni/Ti/Au metal stack, is an anomalous alloy extension of the ohmic metal. This was found to be resulted from rapid diffusion of Au. It is demonstrated that by removing the Au under the Ti barrier layer good contact resistance of less than 0.1 Ωmm can be obtained without anomalous alloy extension.

InSb is one of the most promising candidates for the channel layers of the post scaling LSIs, because it features highest electron mobility of 78,000 cm2/(Vs) and highest saturation velocity of 5×107 cm/s among III–V compound semiconductors. High performance HEMTs based on InSb/InAlSb material system have been already demonstrated [1]. However, growth of high quality InSb on Si is difficult due to the large lattice mismatch of 19.3%. We have recently demonstrated that good InSb epitaxial films can be grown on Si (111) substrates using novel growth technique [2]. This technique is based on the fact that the InSb layer grown on a Si (111) substrate is rotated by 30 degrees with respect to the substrate under a certain initial condition, which reduces the lattice mismatch to only 3.3 %, as shown in Fig. 1. Very high mobility of 40,000 cm2/(Vs) with low carrier concentration of 1.8×1016 cm−3 has been obtained using this growth technique for 1-µm-thick films [3]. Moreover, this drastic reduction of the lattice mismatch produces a new possibility, a pseudomorphic InSb/Si quantum well (QW) MOSFETs based on an ultra thin InSb layer grown directly on Si. When the thickness of the InSb channel layer reduces to the critical thickness, such thin channels should have good quality. This has significant advantages, such as elimination of the time-consuming thick buffer layer growth, good electron confinement by the InSb/Si heterojunction. Furthermore it can receive benefits of SOI if one uses SOI substrates. In this paper, we report the fabrication and the properties of Al 2 O 3 /InSb/Si QW MOSFETs having an ultra thin InSb channel layer.

We investigated the effects of initial In coverage for the preparation of InSb bilayer on electrical properties of InSb films grown by surface reconstruction controlled epitaxy. The electron mobility of the InSb films was affected by the initial In coverage of the In-induced surface reconstruction on Si(111) surface. Electron mobility increased with the increase in the initial In coverage up to 1.5 monolayers (ML), and decreased with further increase in In coverage. The InSb film grown with an optimal initial In coverage of 1.5 ML has a high electron mobility of about 40,000 cm2/(V·s) at room temperature. This may be due to the reduction of the 2×1-Sb surface phase or In islands on the surface after the preparation of the InSb bilayer, which cause dislocations in the film. Therefore, the perfectness of the order of atomic planes in Si–Sb–In is very important for a uniform InSb/Si interface formation before the subsequent InSb molecular beam epitaxy (MBE) growth.

The n-type InSb films were prepared on Si(111) substrates with a two-step growth method via an InSb bilayer. This growth method consists of an initial low-temperature InSb layer growth and a subsequent high-temperature InSb layer growth. In order to obtain a heteroepitaxial InSb film with a high electron mobility, the growth conditions of the first InSb layer were optimized. The first InSb layer was prepared at higher growth temperatures. Moreover, the thickness of the first InSb layer with a lower crystalline quality and poor electrical properties decreased. InSb films prepared with new deposition conditions showed a higher crystalline quality, a lower defects density, and better electrical properties than the films indicated in our previous report. An InSb film with a high electron mobility of 38,000 cm2/(V·s) which shows a high potential for new high-speed device applications was obtained.